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Abstract. - In this letter I present results from a correlation analysis of three galaxy redshift 
catalogs: the SSRS2, the CfA2 and the PSCz. I will focus on the observation that the amplitude 
of the two-point correlation function rises if the depth of the sample is increased. There are 
two competing explanations for this observation, one in terms of a fractal scaling, the other 
based on luminosity segregation. I will show that there is strong evidence that the observed 
^ ■ growth is due to a luminosity dependent clustering of the galaxies. 
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Oh' Introduction. - One of the problems in cosmology is to understand the formation of the 

large-scale structures in the Universe, as traced by the spatial distribution of galaxies. The- 
oretical models of large-scale structure and galaxy formation, whether involving analytical 
predictions or numerical simulations, are based on some form of random or stochastic initial 
conditions. This means that a statistical interpretation of the observed galaxy distribution 
is required, and that statistical tools must be deployed in order to discriminate between dif- 
ferent cosmological models (e.g. pP). The most frequently employed statistical measure is 
the two-point correlation function. Higher-order correlations are important, but already the 
observed two-point correlation properties of the galaxy distribution impose strong constraints 
on the models of structure formation. However, two basically different interpretations of the 
observed two-point properties are discussed: in the "standard" picture the galaxy distribution 
is assumed to be homogeneous on large scales. The correlations of the deviations from this ho- 
mogeneous distribution are quantified by the two-point correlation function £(r) (see e.g. 
In the alternative picture the galaxy distribution is modelled as a fractal. The two-point 
correlations are then measured with the conditional density T(r) (see e.g. The inhomo- 
geneous nature of a fractal challenges the standard picture. Clearly, these two models lead to 
different interpretations of observational results. In this letter I will focus on the growing of 
the amplitude of the two-point correlation function £(r) with the sample depth. This growing 
amplitude is either explained with the scaling properties of a fractal or with luminosity seg- 
regation, a luminosity dependent clustering strength. By reanalysing three galaxy catalogues 
I will show that there are strong arguments in favour of luminosity segregation. 
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Fig. 1 - Luminosity L of the galaxies in the Southern Sky Redshift Survey 2 (SSRS2) plotted against 
their distance s to our galaxy. Volume-limited samples comprise the galaxies in the upper left part 
enclosed by the solid lines (120/i _1 Mpc), dotted lines (100/i _1 Mpc), short dashed lines (90/i _1 Mpc), 
long dashed lines (80/i _1 Mpc), and dashed-dotted lines (70/i -1 Mpc). Low luminosity galaxies at large 
distances are not observed, as can be seen from the empty region in the lower right part. 

Fig. 2 - Sample geometry of a volume- limited sample with sample depth R (simplified sketch). Only 
galaxies inside the opening angle Q and with a distance s < R enter the sample. 

Two-point correlations. - Let me first discuss the stochastic picture where the galaxies 
positions in space are treated as a realization of a random process. The product density 
p 2 (xi, X2)dF(xi)dF(x 2 ) is the probability of finding two galaxies in the volume elements 
dF(xi) and dT^(x 2 ), respectively. In a homogeneous and isotropic point distribution with 
mean number density p one defines the two-point correlation function £(r) (e.g. 



with r = |xi — x 2 |. The conditional density can be defined as T(r) = p(S,(r) + 1). For a point 
distribution on a fractal the mean number density p, and also £(r) are not well defined. Thus, 
in this case, one investigates the two-point correlations with the conditional density T(r): 
the density of galaxies at a distance of r as seen from another galaxy 0j. A scale invariant 
cumulant £(r) is typically found in critical systems, whereas a scale invariant conditional 
density T(r) is an indication for a fractal system. Clearly, only in the limit r — > 0, both £(r) 
and r(r) may show the same scaling behaviour. An instructive discussion of the different 
scaling regimes in the galaxy distribution is presented by 

Galaxy samples. - In a typical galaxy catalogue the position on the sky inside a given 
angular region and the flux in a given wave-band are measured. The distance to our position 
is estimated from the redshift of the galaxy. A flux-limited sample consists out of galaxies 
down to a limiting flux /i; m . To study the clustering properties of such a galaxy catalogue, I 
extract a sequence of volume-limited samples. A volume-limited subsample is constructed by 
introducing a limiting depth R and a limiting luminosity Ln m and by admitting only galaxies 
within a distance s < R from our position and a luminosity L > Ly lm (L\ lm oc R /ii ro , in the 
Euclidean case, see Fig.^). For a given sample one defines the sample-dependent number den- 
sity ps = N/V, with the number of galaxies N, and the volume V. The two-point correlation 
function £ s , and the conditional density T s determined from this sample satisfy the relation 
Ts( r ) — PsO- + £s( r ))- If our galaxy sample stems from a homogeneous distribution, neither 
£s nor Tg should change if one increases the sample size (despite statistical fluctuations). If 



/ 9 2 (xi,x 2 )= j o 2 (e(r) + f). 



(1) 
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Fig. 3 - £(r) and T(r) for volume limited samples from the SSRS2 with depth 120/i *Mpc (solid 
line), 100h _1 Mpc (dotted line), 90/i _1 Mpc (short dashed line), 80h _1 Mpc (long dashed line), and 
70/i _1 Mpc (dashed-dotted line). The limiting luminosity changes in these samples. The labels LS and 
minus mark the results obtained with the Landy & Szalay and the minus estimator, respectively. The 
solid dot marks ro(120/i _1 Mpc), the open dots mark ro(lQO/i _1 Mpc), r (90/i _1 Mpc), ro(80/i _1 Mpc), 
and ro(70/i _1 Mpc) from right to left, according to Eq.@. The smooth solid line is £(r) for a fractal 
with D — 2 and ro(70/i _1 Mpc) according to Eq.J^Jl. Two-cr error bars, determined from a Poisson 
process, are shown only for the 120/i~ Mpc and 70/i _1 Mpc samples. 



our sample stems from a point distribution on a fractal the number density ps is depending 
on the sample size, but the scaling exponent D — 3 of Ts(r) oc r D ~ 3 stays invariant (D is the 
correlation dimension). As a result £s is changing with the size of the sample (see e.g. [3]). For 
the two-point correlation function often the following parameterisation is used £s(r) = (^-) 7 
with the scaling index 7. The so-called "correlation length" r$ quantifies the amplitude 
of a scale invariant correlation function. Consider a large sample with a depth i? ma x and 
several smaller samples R < R max within (see Fig. 0) . On a fractal r is proportional to R 6 , 
specifically 

R ( r \ D ~ 3 

r (R) = roCiW), and £(r) = 2 —— - 1. (2) 

-rtmax \ro{R)J 

In the following I will estimate the two-point correlation function as well as the conditional 
density for three galaxy samples. I checked that for the samples and the scales considered 
here, the estimators discussed by [7| give consistent results. I illustrate this by showing the 
results for £(r) both for the minus (reduced sample) estimator as favoured by 0, and the 
estimator due to Landy & Szalay [7] showed that the Landy & Szalay estimator has 
preferable variance properties. 

Luminosity segregation but no fractal scaling in the SSRS2. - The Southern Sky Redshift 
Survey 2 (SSRS2, 9 ) is 99% complete with a limiting magnitude of mn m = 15.5 (magnitudes 
are logarithmic flux measures). The angular extent is —40° < 6 < —2.5° with b < —40° and 
5 < 0° with b > 35° (declination 5, galactic latitude b). The magnitudes were if -corrected 
as described in JOjj and luminosity distances were used. Nearly identical results could be 
obtained using Euclidean distances and no if-correction. In Fig. [3] both £(r) and T(r) are 
shown for a sequence of volume-limited samples from the SSRS2. The number density in 
the volume-limited samples decreases from 70/i _1 Mpc to 120/i _1 Mpc. Consequently, the 
conditional density T(r) is decreasing with the sample depth. The amplitude of £(r) increases 
with the depth of the samples, and r roughly follows the relation (J2J. This was interpreted 
as a sign of a fractal galaxy distribution (e.g. 3 ). However, another explanation is possible. 
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Fig. 4 - f (r) and F(r) for subsamples from the volume-limited samples from the SSRS2 with a depth 
of _R max = 120/i _1 Mpc. The depth of the sub-samples is 120/i _1 Mpc (solid line), 100/i _1 Mpc (dotted 
line), 90/i _1 Mpc (short dashed line), 80/t _1 Mpc (long dashed line), and 70/i -1 Mpc (dashed-dotted 
line). All samples have the same limiting luminosity. The smooth solid line is £(r) for a fractal with 
D = 2 and r {70h~ 1 Mpc) according to Eq.@. Marks as in Fig. 01 



Due to the construction of the volume-limited sample the mean absolute luminosity of the 
galaxies in the sample increases with the depth of the sample (compare with Fig. Hence, 
the growing amplitude of the correlation function for deeper volume-limited samples may be 
a result of the stronger clustering of the more luminous galaxies. This is called luminosity 
segregation. Clearly, the two-point correlation function £(r) applied to a sequence of volume- 
limited samples is not able to distinguish between both claims. To test the scaling relation (|2J) 
independent from any luminosity dependence I use a volume-limited sample with a depth of 
Rmax = 120/i _1 Mpc. From this volume-limited sample I extract a sequence of subsamples with 
depths of R < i? max (see Fig. [2Jl . All these subsamples have the same lower limit in luminosity 
(see Fig. nj. As can be seen from Fig. 0] the estimated £(r) are consistent in these samples 
but inconsistent with the fractal prediction. There is no indication for a fractal scaling of 
the "correlation length" ro(R) as given in Eq.(J2J). Moreover, the conditional densities T(r) of 
these samples nearly overlap. Measurement errors, e.g. for the position of the galaxies, have no 
visible effect on the correlation functions. The dominant contribution is the statistical error. 
However there is only one realisation of the galaxy distribution in the Universe. Therefore, 
I have to assume a model to quantify the statistical errors. The simplest model is a purely 
random distribution of points, the Poisson process. I estimate the errors from 100 realisations 
of a Poisson process with the sample geometry, and the number density as in the galaxy 
samples. Later on I will show that a more realistic modelling leads to larger errors. These 
errors are within the same order as determined from a Poisson process. For both models, the 
errors are smaller than the predicted effects from fractal scaling. 

Luminosity segregation but no fractal scaling in the CfA2. - As a spatially complemen- 
tary sample to the SSRS2 I use a galaxy sample from the CfA galaxy catalogue (see aim 
references therein), with 5 > 0, and \b\ > 35 and a limiting magnitude of mn m = 15.5. From 
this galaxy catalogue I extract similar volume-limited samples as for the SSRS2. The results 
shown in Figs. 15161 lead to the same interpretation as for the SSRS2: the growing amplitude 
of £(r) is caused by luminosity segregation. 

Neither luminosity segregation nor fractal scaling in the PSCz. - Both the galaxies in 
the SSRS2 and the CfA2 were selected in the optical waveband. The galaxies in the PSCz 
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Fig. 5 - f (r) for a sequence of volume-limited samples with changing limiting Luminosity Ln m from 
the CfA2 determined with the Landy&Szalay estimator. The smooth solid line is f (r) for a fractal 
with D = 2 and r- (70/t _1 Mpc) according to Eq.J5J. Marks as in Figs. 

Fig. 6 - £(r) for samples with varying depth but with the same limiting luminosity extracted from a 
volume-limited sample of the CfA2 with a depth of 7i max = 120/i -1 Mpc. Marks as in Fig. 2] 



survey were selected according to their flux in the infrared as detected by the IRAS satellite. 
A detailed description of the IRAS PSCz galaxy catalogue may be found in ^21- I extract 
volume-limited samples from the PSCz survey using luminosity distances within the standard 
masked area. I approximate the sample geometry by two spherical caps with galactic latitude 
b > 5° for the northern part and with b < —5° for the southern part. Hence, I neglect some 
regions which were left empty due to galactic absorption or confusion in the IRAS PSC maps. 
I filled these empty regions with random points assuming the same number density as in 
the fully sampled region. No differences in the correlation properties between the filled and 
unfilled samples are visible in the two-point measures. The £(r) determined from a sequence 
of volume-limited samples is inconsistent with the fractal prediction and shows no significant 
variation of tq with the sample size (Fig. [7J|. As expected, extracting subsamples from one 
volume-limited sample with i? max = 120/i _1 Mpc does not change this behaviour, although the 
fluctuations increase in the sparser samples (Fig.[8J). Clearly, there is neither an indication for 
a fractal scaling of ro with the sample depth nor for luminosity segregation (see also 221)- Due 
to the selection of galaxies in the infrared one does miss early type (e.g. elliptical) galaxies. 
Because of this selection one does not find luminosity segregation in the PSCz |T3]. To go 
beyond the error estimates relying on the Poisson process I estimate the errors for £ (r) from the 
fluctuations between eleven mock galaxy catalogues, constructed from an TV-body simulation 
based on a ACDM cosmology( 1 ). For such a clustered point distribution, the error bars are 
larger than in the case of the Poisson process, but still of the same order (see Figs. 17181) . 
In both models, these errors are smaller than the predicted effects of fractal scaling on £(r) 
(two-cr error bars are shown in the plots). 

Summary. - By analysing three different galaxy catalogues I could show that the ampli- 
tude of the correlation function £(r) is depending on the luminosity. Luminosity segregation 
already has been found in the SSRS2, CfA2 and also the 2dF and the SDSS galaxy catalogues 
(see e.g. |16II17II18II10| [19 20 , 21 ). In these investigations mainly volume-limited samples with 
varying depths or directly flux-limited samples have been used. Using such samples one is 



( 1 )A description of the procedure and references to articles describing the simulation and the construction of 
the mock catalogues can be found in II 51 . 
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Fig. 7 - £(r) for a sequence of volume- limited samples with changing limiting luminosity Ln m from 
the PSCz determined with the Landy & Szalay estimator with marks similar to Fig. [3] The smooth 
solid line is £(r) for a fractal with D — 2.2 and ro(70/i _1 Mpc) according to Eq.J^J. Two-ct error bars 
are shown only for the 120/i~ 1 Mpc sample. The larger error bars were determined from the mock 
catalogues. The Poisson error bars are slightly shifted to the left. 

Fig. 8 - £(r) for samples with varying depth but with the same limiting Luminosity Ln m extracted 
from a volume-limited sample of the PSCz with a depth of i? max = 120ft _1 Mpc similar to Fig. 
Marks as in Fig. |7| but the two-er error bars are shown only for the 70/i _1 Mpc sample. 



not able to separate the influence of luminosity segregation from a possible fractal scaling. In 
samples with a fixed lower limit in the luminosity and then reducing the depth of the samples 
I found no significant changes in the correlation function. Specifically, I found no sign for a 
growth of the correlation length ro with increasing sample depths in these cases. This is a 
strong indication that the growth of ro in standard volume-limited samples is caused by lumi- 
nosity segregation, and a fractal explanation is disfavoured. Using a similar construction [3] 
found a growing ro in the Perseus Pisces survey (PPS). Our consistent results, both from the 
significantly larger SSRS2 and the CfA2 indicate that the PPS is too small in size to give 
reliable results. In the CfA2 [22| found also no significant growth of ro using a comparable 
method. ^3] used mark correlation functions to quantify the luminosity dependency of the 
galaxy clustering in the SSRS2 in a scale dependent way, uninfluenced by inhomogeneities. 

I limited my investigations to the question: what causes the growing amplitude of the 
two-point correlation function? Already with the current galaxy catalogues one is able to 
show that the amplitude of the two-point correlation function is depending on the luminosity 
of the galaxies. If one a priori assumes that the galaxy distribution is a fractal, these results 
may be interpreted as large fluctuations, which indeed are common in fractals. However, these 
fluctuations must conspire in all the samples considered here, to give the observed result of a 
constant amplitude of the two-point correlation function. Fluctuations in the morphological 
properties of the large scale distribution of galaxies have been detected out to a scale of 
200/i -1 Mpc (TS|. However, these fluctuations are barely visible with two-point measures, 
and they are still compatible with fluctuations expected from a ACDM model. I did not 
comment on the topic, whether one already does see a turnover to homogeneity from current 
galaxy surveys |15j . and whether £(r) or T(r) shows the more extended scaling regime. For a 
discussion see |23I5| . I expect that the completed Sloan Digital Sky Survey will offer conclusive 
evidence for these points. 
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